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We sought to identify those institutions helping students overcome barriers to college completion 
and achieve a livable wage. Our analysis of Illinois 4-year postsecondary institutions highlights those 
institutions which fostered degree completion and job success with less debt for underrepresented 
minority students, first-generation students, and low-income students. Combined, we define this 
group as traditionally underserved students. 


Background 


There is extensive information available for prospective college students to evaluate colleges, 
including various college rankings. This abundance of college information and ranking systems 
has the potential to ensure that all students find the best college fit. Yet, among those less 
familiar with college (e.g., first-generation college students), college choice often depends 
on a small set of practical concerns, which tends to limit their postsecondary choices. This 
study focuses on some of the most relevant variables for later financial success: graduation 
rates, including for minority students; earnings after graduation; and student debt and default 
rates. We also devised a ranking system that emphasized Illinois institutions that are strong 
academically, but are attainable and realistic options for students who have average ACT scores 


and high school GPAs. 


Purpose 


We applied the ranking methodology used in Holt, White, and Terrell (2017), with some 
adaptations, to generate a new metric and identify the top 4-year postsecondary institutions 
in Illinois for graduating underserved students and transitioning them to employment, while 
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helping students to accrue minimal debt. By creating a tailored or personalized ranking formula, 
we were better able to identify those institutions that are leading the way and fostering success 
for underserved students. We also investigated correlates of these success metrics among these 
universities and colleges. 


Methods 


Sample 


Illinois 4-year colleges and universities that primarily grant bachelor’s degrees, as identified by 
the Integrated Postsecondary Education Data System (IPEDS), were included in this study. In 
total, there were 55 four-year colleges or universities including 4-year public (n=11), private 
nonprofit (n=38), and private for-profit (n=6) 4-year colleges and universities from across the 


Northeast 
rs 


state in our analyses (see Figure 1). 


Northwest 


Chicago 


* Top 7 Institutions 
@ Institutions 8-31 
@ Institutions below thresholds 


Figure 1. Map of the universities included in the study with the Top 7 indicated by blue stars. 
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Data and Metrics 


Most data used originated from the 2013-2014 and 2014-2015 College Scorecard’ and IPEDS 
data files, with a few variables from 2012-2013 data file.° The two most recent years of available 


2 


data for each variable were averaged prior to analysis. Averages between multiple years were not 
computed for variables that were pooled and/or already contained multiple years of data (e.g., 
three-year cohort default rates). 


‘The variables of interest were sorted into three categories: 


* Categorical Predictors including sector, locale size, locale type, and region within the 
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state. 


¢ Continuous predictors including three sub-categories of variables that consisted of 
undergraduate student enrollment, cost, and percentages of degrees awarded by field.“ 
‘These degrees by major field variables were computed from the “percentages of degrees 
awarded in...” from IPEDS collected via College Scorecard. 


* Outcome variables including student retention rate, 6-year graduation rate for all 
students, 6-year graduation rate for Black students, 6-year graduation rate for Hispanic 
students, median debt for graduates, 3-year cohort default rate (CDR), and median 
earnings after 10-years after entry. 


Analysis and Results 


Phase 1 


Multiple regression analyses were conducted, regressing each of the seven outcome variable 
on four predictors representing the underserved student context: the percentage of Black 
students, the percentage of Hispanic students, the percentage of first-generation students, and 
the percentage of Pell Grant recipients. The residuals were outputted for each regression model 
and used for further analysis. These residuals indicate the deviation of the actual outcome value 
from the predicted value, given a specific underserved student context. This step was designed 
to remove the advantage in the outcome variables that institutions might have by admitting 
fewer underserved students. 


Phase 2 


Using principal components analysis to generate a smaller number of composite variables 
which underlie the covariance among the outcome variables, we identified a 2-factor solution 
which accounted for 55% of the variance. Five variables loaded on component one: retention 


: https://collegescorecard.ed.gov/data/ 
e https://nces.ed.gov/ipeds/Home/UseTheData 


* Data from 2012-2013 were data from the National Student Loan Data System that were not in IPEDS and were the most 
recent data available at time of analysis. 


‘ STEM definition from Nash and Zaback (2011); defined by following degree areas: agriculture, agriculture operations, and 
related services; natural resources and conservation; architecture and related services; computer and information sciences and 
support services; engineering; engineering technologies and engineering-related fields; biological and biomedical sciences; 
mathematics and statistics; physical sciences; science technologies/technicians 
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rate; 6-year completion rates overall, for Black students, and for Hispanic students; and median 
earnings 10-years after entry. This component was named Success. The variables that loaded on 
the second component included median debt at graduation and cohort default rates and this 
component was named Debt. ‘The rotated structure matrix is given in Table 1. 


Table 1 
Rotated Loading Matrix 


Component 


NETat-1¢) (=) 1 (Success) yt (BY-) 0) 9) 


6-yr. Completion Rate for Hispanic Students 0.16 


Student Retention Rate 0.58 -0.14 


3-yr. Cohort Default Rate -0.15 0.83 


The scores on component two, Debt, were subtracted from the scores on component one, 
Success, to create an overall score, which represented high success with low debt. We termed 
the overall score, Return-on-Investment (ROI). We assume that higher ROI scores were due to 
internal processes but not institutional selectivity. 


Component scores were tested for associations with institutional variables. We found a 
statistically significant association of Debt and sector of the university, F (2, 52) = 5.16, p = 
.009, 7° = .17. Specifically, for-profit universities had significantly higher scores on the Debt 
component than public institutions. University of Illinois Chicago (UIC) was an extreme 
outlier with low debt, while Northeastern Illinois University was a low outlier on Success. 
Finally, UIC was a high outlier on ROI among the public institutions and Columbia College 
was a low outlier on ROI among the private nonprofit colleges. 


Interestingly, we also found that a disproportionately higher number of underserved students 
attended for-profit institutions. This held true for racial/ethnic minority students, low-income 
students, first-generation students, and non-traditionally-aged students (see Table 2). This was 
most evident for low-income students, who constituted 69% of the for-profit enrollment, 
compared to 38% and 42% of nonprofit and public institution enrollment, respectively. 


Table 2 
Population of Underserved Students by Institution Sector 


ba =) % Students 
% Black % Hispanic Generation % Pell Grant Aged 225 


ASY-Tei fo) g Students Students Students Recipients Rate 
Public (N=11) 


Private Nonprofit (N=38) 
Private For-Profit (N=6) 


Total (N=55) 


Source: College Scorecard, AY 2014-15 sector (control of institution); total share of enrollment for Black and Hispanic undergraduate 
students; AY 2013-14 & 2014-15 pooled cohorts of 1st generation students; average of AY 2012-13 & 2013-14 total undergraduate 
recipients of a Pell Grant, fall 2013 percentage of undergraduates aged 25 and above, no other years available; sector is from AY 
2014-15. 
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Phase 3 


In the third phase, thresholds were applied to the institutions and they were ranked. The 
thresholds were designed to ensure that those highly ranked institutions were serving a 
moderate number of underserved students and had a reasonably high overall graduation rate. 
Accordingly, we required institutions to have both an overall 6-year graduation rate of at least 
50% and at least 25% Pell Grant recipients enrolled. 


‘These two thresholds removed 24 institutions of the 55 from the final analysis, resulting in 31 
institutions that could be ranked. The overall ROI scores of the remaining 31 institutions were 
calculated and rank ordered. The cut off for the top performing group was drawn between the 
7th and 8th ranked institutions because there was a small gap between the 7th institution and 
the next cluster. The top ranked seven institutions are given in Table 3. 


Table 3 
Top 7 Illinois 4-Year Institutions 


Abbreviated 


Institution Name 


Illinois Institute of Technology 


Saint Xavier University 


DePaul University DePaul 1.4 


Table 3 depicts the rank of the Top 7 institutions as defined by their overall ROI score, as well 
as their name and abbreviation. The top institution, an outlier with an overall ROI score of 
4.1, is the University of Illinois Chicago. The remaining six institutions, Illinois Institute of 
Technology, Elmhurst College, Saint Xavier University, Eastern Illinois University, DePaul 
University, and Western Illinois University, each had an ROI score between 1.5 and 1.0. 


Location of Top 7 


¢ Four of the Top 7 (57.1%) fall within a city, one within a suburb, and two within a 
town or rural area. 


e Five of the Top 7 (71.4%) institutions fall within a large municipality, one is in a 
midsized locale, and one is within a municipality with a small population. 


¢ Four of the seven (57.1%) are located within Chicago, one is located in the 
Northeast, outside of Chicago, one is located in the West Central region, and one is 
in the East Central region. 
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Characteristics of High-Ranking Institutions 


Our research indicates that there are public and nonprofit private institutions that rank highly 
on the ROI metric. The for-profit sector had considerably worse outcomes as a whole than the 
other sectors and none of the for-profit institutions ranked in the Top 7. Public institutions 
made up 11 of the 55 institutions (20%) included in our analysis but three of institutions in 
the Top 7 (43%). This over-representation suggests that M/linois public universities are successfully 
fulfilling their missions of providing affordable educational options for students, including those 
traditionally underserved. 


Cornerstones of Student Success 


It is important to note that the institutional sector with the highest proportion of underserved 
students was the for-profit private sector. Yet, none of the Top 7 or 31 ranked institutions were from 
this sector and the outcome variables were considerably less positive for this sector. This indicates 
a mismatch between the type of institution that most benefits underserved students according 
to our study and the institutions with the highest enrollment of underserved students. 


Of the cost variables we analyzed, the only ones that were consistently lower for the Top 7 than 
the other tiers of institutions were net price overall and net price for those students from the 
lowest income families. These variables are probably the most salient for underserved students, 
certainly for students from low-income families. It is also interesting to note that CDRs were 
dramatically higher for for-profit private institutions. Cohort default rates are a reflection of 
institutional affordability to the students post-graduation. If college costs are high but job 
placement and salaries are high, we would not expect high CDRs. 


Limitations 


1. This study was limited by availability of public data from IPEDS, which currently 
does not report graduation rates for part-time, non-traditional, and graduate students. 
Further, limitations on earnings data include a lack of information on students currently 
in graduate school and these data are not representative of institutions with a low 
proportion of Title IV-eligible students (College Scorecard, 2017). 


2. We selected Illinois institutions with a physical presence for this study. As higher 
education is increasingly moving online, there may be online higher education 
institutions which serve a large number of students in Illinois that were not listed in 
IPEDS or College Scorecard as Illinois institutions. 


3. Rankings are related to this study’s purpose and assumptions. We acknowledge that 
these rankings are directly related to this study’s purpose and assumptions and rankings 
with different purposes or assumptions would yield different results. 
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Implications for Policy and Practice 


1. Personalize Rankings for a Good Institutional Match. College ranking systems have 
different methodologies, metrics, assumptions, and audiences. High school counselors, 
students, teachers, parents, and access organizations should consider matching the 
students’ characteristics with the success characteristics of the institution. 


2. Identify and Recognize Institutions Supporting Underserved Students in their 
Community. To expand affordable and promising higher education options to 
underserved populations it is important to identify successful strategies and to replicate 
and adapt these strategies at similar institutions. 


3. Increase College Affordability. Advocacy is needed to keep tuition and fees proportional 
to increases in cost of living at both private and public universities, so that college costs 
do not price out underserved students from the college market. In Illinois, the state 
need-based financial aid program, the Monetary Award Program (MAP), has not kept up 
with increases in tuition and fees and therefore it does not close the gap for low-income 
students, as it did in past years. If fully resourced, however, the MAP program could play 
an important role in closing the tuition gap for low-income students in Illinois. 
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The Illinois Education Research Council at Southern Illinois University Edwardsville was 
established in 2000 to provide Illinois with education research to support Illinois P-20 education 
policy making and program development. The IERC undertakes independent research and policy 

analysis, often in collaboration with other researchers, that informs and strengthens Illinois’ 
commitment to providing a seamless system of educational opportunities for its citizens. Through 
publications, presentations, participation on committees, and a research symposium, the IERC 
brings objective and reliable evidence to the work of state policymakers and practitioners. 


The full report is available at http://ierc.education 
For further information, contact the IERC at Southern Illinois University Edwardsville 


toll-free at 1-866-799-IERC (4372) or by email at ierc@siue.edu 


